Hierarchical Semi-Sparse Cubes-Parallel Framework for Storing Multi-Modal Big Data in HDF5

0581668 - ASÚ 2024 RIV US eng J - Článek v odborném periodiku
Nádvorník, J. - Škoda, Petr - Tvrdík, P.
Hierarchical Semi-Sparse Cubes-Parallel Framework for Storing Multi-Modal Big Data in HDF5.
IEEE Access. Roč. 11, November (2023), s. 119876-119897. ISSN 2169-3536. E-ISSN 2169-3536
Institucionální podpora: RVO:67985815
Klíčová slova: big data * multi-modal data * multi-dimensional data
Obor OECD: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Impakt faktor: 3.9, rok: 2022
Způsob publikování: Open access

In this article, we present a revised version of the Hierarchical Semi-Sparse Cube (HiSS-Cube) framework. It aims to provide highly parallel processing of combined multi-modal multi-dimensional big data. The main contributions of this study are as follows: 1) Highly parallel construction of a database built on top of the HDF5 framework. This database supports parallel queries 2) design of a database index on top of HDF5 that can be easily constructed in parallel 3) support of efficient multi-modal big data combinations. We tested the scalability and efficiency on big astronomical spectroscopic and photometric data obtained from the Sloan Digital Sky Survey. The performance of HiSS-Cube is bounded by the I/O bandwidth and I/O operations per second of the underlying parallel file system, and it scales linearly with the number of I/O nodes.
Trvalý link: https://hdl.handle.net/11104/0349775