Back to Search Start Over

Spotting Code Optimizations in Data-Parallel Pipelines through PeriSCOPE.

Authors :
Fan, Xuepeng
Guo, Zhenyu
Jin, Hai
Liao, Xiaofei
Zhang, Jiaxing
Zhou, Hucheng
McDirmid, Sean
Lin, Wei
Zhou, Jingren
Zhou, Lidong
Source :
IEEE Transactions on Parallel & Distributed Systems; Jun2015, Vol. 26 Issue 6, p1718-1731, 14p
Publication Year :
2015

Abstract

To minimize the amount of data-shuffling I/O that occurs between the pipeline stages of a distributed data-parallel program, its procedural code must be optimized with full awareness of the pipeline that it executes in. Unfortunately, neither pipeline optimizers nor traditional compilers examine both the pipeline and procedural code of a data-parallel program so programmers must either hand-optimize their program across pipeline stages or live with poor performance. To resolve this tension between performance and programmability, this paper describes PeriSCOPE, which automatically optimizes adata-parallel program’s procedural code in the context of data flow that is reconstructed from the program’s pipeline topology. Such optimizations eliminate unnecessary code and data, perform early data filtering, and calculate small derived values (e.g., predicates) earlier in the pipeline, so that less data—sometimes much less data—is transferred between pipeline stages. PeriSCOPE further leverages symbolic execution to enlarge the scope of such optimizations by eliminating dead code. We describe how PeriSCOPE is implemented and evaluate its effectiveness on real production jobs. [ABSTRACT FROM PUBLISHER]

Details

Language :
English
ISSN :
10459219
Volume :
26
Issue :
6
Database :
Complementary Index
Journal :
IEEE Transactions on Parallel & Distributed Systems
Publication Type :
Academic Journal
Accession number :
102658366
Full Text :
https://doi.org/10.1109/TPDS.2014.2326416