Back to Search Start Over

PyOphidia: A Python library for High Performance Data Analytics at scale

Authors :
Donatello Elia
Cosimo Palazzo
Sandro Fiore
Alessandro D’Anca
Andrea Mariello
Giovanni Aloisio
Source :
SoftwareX, Vol 24, Iss , Pp 101538- (2023)
Publication Year :
2023
Publisher :
Elsevier, 2023.

Abstract

The increasing size of scientific datasets has caused a deep transformation in the way scientific research is currently carried out. In multiple domains, Big Data challenges have called for novel software solutions capable of exploiting fast computing resources, workflows technologies and parallel paradigms at scale, by providing a high-level of abstraction while hiding, at the same time, the underlying infrastructural and software complexity from scientists. This paper describes PyOphidia, an open-source Python module for High Performance Data Analytics on multi-dimensional scientific datasets. PyOphidia aims to simplify the execution of parallel data analysis over scientific datacubes on High Performance Computing infrastructures. It can be easily integrated with other existing Python libraries and tools, within multiple data science environments.

Details

Language :
English
ISSN :
23527110
Volume :
24
Issue :
101538-
Database :
Directory of Open Access Journals
Journal :
SoftwareX
Publication Type :
Academic Journal
Accession number :
edsdoj.b22b1e31ff9149158983161472f74b9f
Document Type :
article
Full Text :
https://doi.org/10.1016/j.softx.2023.101538