1. Data Life Aware Model Updating Strategy for Stream-Based Online Deep Learning
- Author
-
Donglin Yang, Wei Rang, Yu Wang, and Dazhao Cheng
- Subjects
Online model ,020203 distributed computing ,Distributed database ,Computer science ,business.industry ,Deep learning ,Training (meteorology) ,02 engineering and technology ,Machine learning ,computer.software_genre ,Continuous training ,law.invention ,Data modeling ,Computational Theory and Mathematics ,PageRank ,Hardware and Architecture ,law ,Middleware ,Signal Processing ,0202 electrical engineering, electronic engineering, information engineering ,Artificial intelligence ,business ,computer - Abstract
Many deep learning applications deployed in dynamic environments change over time, in which the training models are supposed to be continuously updated with streaming data to guarantee better descriptions of data trends. However, most state-of-the-art learning frameworks support well in offline training methods while omitting online model updating strategies. In this work, we propose and implement iDlaLayer , a thin middleware layer on top of existing training frameworks that streamlines the support and implementation of online deep learning applications. In pursuit of good model quality and fast data incorporation, we design a Data Life Aware model updating strategy (DLA), which builds training data samples according to contributions of data from different life stages, and considers the training cost consumed in model updating. We evaluate iDlaLayer's performance through simulations and experiments based on TensorflowOnSpark with three representative online learning workloads. Our experimental results demonstrate that iDlaLayer reduces the overall elapsed time of ResNet, DeepFM and PageRank by 11.3, 28.2, and 15.2 percent compared to the periodic update strategy, respectively. It further achieves an average 20 percent decrease in training cost and brings about a 5 percent improvement in model quality against the traditional continuous training method.
- Published
- 2021
- Full Text
- View/download PDF