1. Portable Intermediate Representation for Efficient Big Data Analytics
- Author
-
Vana Kalogeraki, Giannis Tzouros, Michail Tsenos, Department of Informatics [Athens], Athens University of Economics and Business (AUEB), Miguel Matos, Fabíola Greve, TC 6, and WG 6.1
- Subjects
Data processing ,Database ,Grammar ,business.industry ,Computer science ,media_common.quotation_subject ,Big data ,Process (computing) ,020207 software engineering ,02 engineering and technology ,computer.file_format ,computer.software_genre ,[INFO.INFO-NI]Computer Science [cs]/Networking and Internet Architecture [cs.NI] ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Data analysis ,[INFO]Computer Science [cs] ,Compiler ,Executable ,Architecture ,business ,computer ,media_common - Abstract
Part 2: Fault Tolerance and Big Data; International audience; To process big data, applications have been utilizing data processing libraries over the last years, which are however not optimized to work together for efficient processing. Intermediate Representations (IR) have been introduced for unifying essential functions into an abstract interface that supports cross-optimization between applications. Still, the efficiency of an IR depends on the architecture and the tools required for compilation and execution. In this paper, we present a first glance at a framework that provides an IR by creating containers with executable code from structures of data analytics functions, described in an input grammar. These containers process data in query lists and they can be executed either standalone or integrated with other big data analytics applications without the need to compile the entire framework.
- Published
- 2021